In recent years, many products that perform encoding using formats standardized by MPEG (ISO/IEC SC29/WG11: Moving Picture Expert Group) have been delivered.
FIG. 6 shows the arrangement of a conventional MPEG encoding apparatus. This example shows the arrangement of an MPEG-2 (ISO/IEC 1178-2) encoding apparatus.
Referring to FIG. 6, reference numeral 601 denotes a motion detection/compensation unit, which detects motion between frames, computes an optimal motion vector, and removes spatial redundancy. Reference numeral 602 denotes an inter-frame differentiator which calculates a difference between frames using the motion vector. Reference numeral 604 denotes a reference frame memory which temporarily stores a reference frame image serving as a reference for motion compensation.
Reference numeral 603 denotes a DCT (Discrete Cosine Transform) unit which performs mutual transformations between the time and frequency domains. Reference numeral 605 denotes a quantizer which quantizes DCT coefficients using a predetermined quantization parameter. Reference numeral 609 denotes a coefficient re-arranging unit which re-arranges the quantized DCT coefficients in a predetermined order to provide an effect of prolonging zero runlength. Reference numeral 610 denotes a DC prediction unit which calculates a difference between DC coefficients of neighboring blocks for respective brightness and color difference components in case of only intra-frame encoding.
On the other hand, reference numeral 608 denotes a dequantizer; 607, an IDCT (inverse DCT) unit; and 606, a frame adder. These units form a local decoder.
Reference numeral 611 denotes a variable-length encoder which performs variable-length encoding of DCT coefficients and the like using Huffman codes.
In the above arrangement, respective processing circuits perform processing in units called macroblocks (to be referred to as MBs hereinafter), as indicated by reference numeral 805 in FIG. 8. Upon 4:2:0 sampling, 16×16 pixels of a brightness component and 8×8 pixels of color difference components are handled as MBs. A block is an element which forms an MB, and includes 8×8 pixels. DCT/IDCT transformation is processed using this unit.
As shown in FIG. 8, in MPEG-2, a bitstream has a structure including a picture 803, group of pictures (GOP) 802, and sequence 801 as upper layers of MBs.
A GOP is formed of a plurality of pictures, as shown in FIG. 7, and I-picture (intra-frame encoding), P-picture (forward predictive coded frame), and B-picture (bidirectionally-predictive coded frame) are used as picture types.
In MPEG-2, a VBV (Video Buffer Verifier) buffer model is defined as the behavior of a decoder, as shown in FIG. 9. This guarantees decodability of respective frames in a decoder, and sets a bylaw that a buffer pointer must always be located between the VBV buffer size (upper limit) and 0 byte (lower limit).
By keeping to this bylaw, the decodability of respective frames in the decoder is guaranteed. If the VBV buffer is lower than the lower limit value, a situation in which a bitstream to be decoded by the decoder within a predetermined time is less than one picture occurs. Hence, the decoder becomes undecodable and frame skip occurs. Such frame skip is inhibited in terms of the standard. FIG. 9 shows such situation.
Therefore, in the prior art, a process for controlling the generated code size in I-picture in correspondence with the remaining VBV buffer size while monitoring an output bit count of respective MBs is done. Also, a method of limiting the resolution of an image to be encoded in accordance with the remaining buffer size to suppress the generated code size has been proposed (e.g., Japanese Patent Laid-Open No. 10-210475).
However, in the prior art, as a measure against VBV buffer underflow, a method of controlling a quantization step is adopted, and cannot reliably prevent underflow for every input.
In a method of preventing underflow by modifying the resolution or the like of a source image, image quality abruptly is impaired at the boundary with a normal operation.